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Abstract 


Complex  products  and  systems  (CoPS)  are  major  capital  goods  in  which 
customers  play  a  central  role  from  design  through  to  disposal,  such  as  large  defense 
equipment  programs  (Davies  &  Hobday,  2005).  The  central  idea  of  the  research 
reported  here  is  that  the  nature  of  complexity  in  CoPS  (here  referred  to  as  Dynamic 
CoPS,  typically  combat  aircraft)  may  have  a  significant  effect  on  the  range  of 
possible  variance  in  their  operations  and  support  (O&S)  costs.  Operational  use  and 
environmental  factors  also  have  an  important  part  to  play  in  the  cost  of  supporting 
Dynamic  CoPS,  which  simple  “parts  count”  approaches  may  miss. 

The  research  is  based  around  a  set  of  case  studies  of  combat  aircraft.  The 
purpose  of  the  case  studies  is  to  explore  how  different  approaches  in  the  U.S.  and 
UK  to  O&S  impact,  and  illustrate,  some  of  the  key  drivers  of  costs  in  O&S. 

Comparisons  of  the  approaches  to  costing  derived  from  the  case  studies  are 
made  to  current  practice  in  the  U.S.  DoD  costing  community,  as  well  as  to  current 
research  on  complexity.  The  findings  are  presented  alongside  recommendations  for 
enhanced  approaches  to  costing  operations  and  support  of  Dynamic  CoPS. 

Keywords:  Costing,  defense  acquisition,  products,  complexity,  operations, 
support,  maintenance,  combat  aircraft,  complex  product  systems,  CoPS,  Dynamic 
CoPS 
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Introduction 


Life  cycle  costing  of  defense  equipment  for  long-term  operations  and  support 
(O&S)  is  extremely  challenging.  The  estimating  of  system  update  costs,  changes  in 
the  roles  and  missions  that  systems  are  used  for,  and  shifts  in  the  commercial  and 
customer  organizations  that  use  and  support  equipment  provide  an  environment  in 
which  major  uncertainties  make  predictions  of  costs  highly  problematic. 

The  research  reported  here  seeks  to  address  these  matters  by  exploring  the 
issues  that  underlie  them,  and  developing  methods  to  complement  and  enhance 
existing  costing  approaches.  These  are  intended  to  help  identify  the  range  of 
variance  in  O&S  costs  from  a  baseline  figure  generated  using  existing  methods.  The 
work  is  structured  in  the  following  three  main  parts: 


an  overview  of  the  key  issues,  the  research  approach  used,  relevant 
literature,  and  the  current  practices  and  knowledge  of  costing  defense 
systems,  and  the  relationships  between  them; 

a  set  of  comparative  case  studies  that  illustrate  the  issues  identified 
and  are  intended  to  provide  the  basic  material  in  identifying  the  nature 
and  scope  of  cost  variance;  and 

a  discussion  of  the  issues  in  light  of  the  case  studies,  and 
recommendations  for  enhanced  approaches  to  costing  in  O&S. 


The  report  is  a  summary  of  the  current  state  of  knowledge  of  the  research, 
and  also  provides  a  basis  for  the  next  stage  in  the  work  (also  funded  by  the  Naval 
Postgraduate  School  [NPS]),  that  will  look  at  the  application,  implementation,  and 
further  refinement  of  the  new  knowledge  of  costing  operations  and  support.  This  is  to 
be  carried  out  under  the  title  Business  Models  for  Cost  Sharing  &  Capability 
Sustainment  and  will  be  reported  in  mid-2012. 
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Section  I.  Overview  and  Background 


A.  Key  Issues  In  Operations  &  Support  Costs 

With  ever-rising  program  costs  and  constant  pressure  on  budgets,  decision¬ 
makers  are  faced  with  a  need  for  the  highest  quality,  robust  cost  estimates  at  the 
start  of  programs  in  order  to  allow  the  best  informed  decisions  to  be  made. 

While  much  work,  over  many  decades,  has  been  focused  on  estimating  the 
costs  of  research  and  development  (R&D),  this  activity  still  poses  problems,  as 
evinced  by  recent  escalations  in  the  Joint  Strike  Fighter  (JSF)  program’s  R&D  cost 
estimates.  However,  an  area  of  even  greater  challenge  is  operations  and  support 
(O&S),  which  is  frequently  the  area  in  which  the  largest  part  of  overall  weapon 
system  life  cycle  costs  reside.  The  unpredictability  of  the  role  of  major  weapon 
systems,  the  environments  in  which  they  will  be  used,  the  multi-decade  duration  of 
their  use,  the  increasing  gaps  between  programs  rendering  analogous  data  “stale,” 
the  extent  and  timing  of  major  platform  upgrades,  changes  in  contractor  business 
models,  and  other  factors  (Kirkpatrick,  1993),  add  up  to  a  series  of  major  challenges 
for  cost  estimators  looking  at  O&S  that  are  common  across  many  nations  and 
endure  over  many  decades. 

The  need  to  make  decisions,  while  still  in  the  early  stages  in  a  program,  that 
ensure  that  force  levels  and  structures  can  be  sustained  over  full  program  lifetimes 
shows  how  understanding  the  degree  of  possible  variance  in  O&S  cost  estimates 
matter — they  can  form  the  greater  part  of  overall  life  cycle  costs  (LCCs).  If  O&S 
costs  turn  out  greater  than  their  estimated  baseline,  then  military  force  structures 
and  capabilities  may  suffer;  legislators  need  to  be  aware  of  any  potential  for  Nunn- 
McCurdy-type  breaches  that  can  lead  to  major  re-planning  of  programs,  with 
attendant  delays.  All  of  these  factors  mean  that  continuous  efforts  need  to  be  made 
to  ensure  that  the  factors  affecting  O&S  costs  are  understood  and  captured  in 
estimates. 
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Currently,  the  approach  used  by  the  U.S.  Department  of  Defense  (DoD)  is 
mandated  through  DoD  Directive  (DoDD)  5000.4  (Deputy  Secretary  of  Defense 
[DEPSECDEF],  2006)  and  implemented  by  the  Cost  Analysis  Improvement  Group 
(CAIG).  The  approach  taken  is  one  of  analytical  cost  estimates,  using  analogies 
from  similar,  older  programs  (where  possible)  to  provide  proxy  data.  A  major 
problem  with  this  approach  is  that  new  technologies  (e.g.,  the  move  from  aluminum 
to  carbon  fiber  structures)  may  make  it  very  difficult  to  “read  across”  old  cost  data 
(Raman,  Graser,  &  Younossi,  2003).  For  some  programs,  it  is  also  possible  to 
provide  “bottom  up”  estimates  using  the  composition  of  more  detailed  cost  estimates 
for  components,  subsystems,  and  so  forth,  to  build  up  an  overall  system  cost  (Arena, 
Younossi,  Brancato,  Blickstein,  &  Grammich,  2008;  Office  of  the  Secretary  of 
Defense-CAIG,  2007).  However,  this  approach  is  often  not  practical  in  the  early 
stages  of  programs,  where  detailed  design  data  are  not  available. 

This  is  of  significance  as  it  is  often  at  the  early  design  stages  that  decisions 
are  made  that  can  have  a  profound  effect  on  O&S  costs.  Basic  configuration 
choices,  such  as  the  use  of  an  existing  or  new  engine;  the  use  of  one  or  two 
engines;  the  use  of  technologies  such  as  low  observables  (stealth)  or  new 
construction  materials/methods  or  computer  system  architectures/software,  are  often 
made  before  any  detailed  costing  is  possible.  That  they  have  a  profound  impact  on 
later  costs  is  borne  out  in  numerous  cases,  with  the  Joint  Strike  Fighter  program 
perhaps  the  most  high  profile  example  of  the  moment.  In  addition,  the  complexity  of 
designing  such  systems,  let  alone  operating  and  supporting  them,  has  led  to 
attempts  by  the  Defense  Advanced  Research  Projects  Agency  (DARPA)  to  redefine 
our  understanding  of  complexity  and  approaches  to  designing  complex  systems. 

1.  The  JSF — O&S  Costs  and  Uncertainty 

The  Joint  Strike  Fighter  serves  as  a  useful  example  of  the  issues  raised  by 
the  effect  of  system  and  program  design  on  costs.  The  whole  purpose  of  the  Tri- 
Service  JSF  is  to  deliver  enhanced  combat  capability  using  new  technologies  at  an 
affordable  price.  According  to  the  official  website  on  the  JSF, 
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The  F-35  Lightning  II  Program  (also  known  as  the  Joint  Strike  Fighter 
Program)  is  the  Department  of  Defense's  focal  point  for  defining  affordable 
next  generation  strike  aircraft  weapon  systems  for  the  Navy,  Air  Force, 
Marines,  and  our  allies.  The  F-35  will  bring  cutting-edge  technologies  to  the 
battlespace  of  the  future.  The  JSF’s  advanced  airframe,  autonomic  logistics, 
avionics,  propulsion  systems,  stealth,  and  firepower  will  ensure  that  the  F-35 
is  the  most  affordable,  lethal,  supportable  and  survivable  aircraft  ever  to  be 
used  by  so  many  warfighters  across  the  globe.  (Joint  Strike  Fighter,  n.d.) 

This  is  a  very  tall  order,  but  from  early  on,  the  assumption  was  that 
affordability  across  the  JSF’s  lifecycle  could  be  realized  by  having  a  large  scale 
program  that  shared  significant  commonalities.  These  commonalities  lay  across 
three  aircraft  variants  developed  by  a  diverse  set  of  companies  in  a  number  of 
nations,  and  used  by  land  and  sea  based  air  arms  around  the  globe. 

While  the  subsequent  travails  of  the  program  have  led  to  significant  increases 
in  the  cost  of  research,  development,  test,  and  evaluation  (RDT&E),  it  is  also  notable 
that  a  fundamental  promise  of  the  JSF — to  provide  higher  levels  of  capability  and 
technology  at  a  similar  or  lower  cost  than  legacy  platforms — appears  likely  to  be 
significantly  breached.  An  assessment  by  the  Naval  Air  Systems  Command  in  2010 
indicated  O&S  costs  40%  higher  than  the  Harrier  and  Hornet  legacy  platforms,  and 
showed  that  “the  ability  to  influence  total  ownership  cost  decrease[d]  over  time” 
(Burgess,  2010).  Although  a  rebuttal  of  the  assessment  was  made  by  the  JSF’s 
prime  contractor,  Lockheed  Martin,  it  is  clear  that  there  is  a  lack  of  agreement  on  the 
JSF’s  O&S  costs  (Trimble,  2010). 

This  is  further  borne  out  by  work  carried  out  in  the  UK  for  an  anonymous  JSF 
customer  nation,  which  revealed  the  inability  of  current  costing  methods  to  give 
anything  other  than  a  wide,  near  100%  variance1  “ballpark”  figure  (Phillips,  2011). 
Phillips’  approach  used  a  “traditional”  one,  with  data  analogies  based  on  older 


1  From  $31  billion  to  $58  billion  for  a  1 00  aircraft  fleet  over  30  years. 
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combat  aircraft  figures.  Phillips  states  that  the  final  JSF  O&S  figures  produced  are 
not  predictions,  but  allow  the  estimator  only  to  “speculate”  at  the  cost  of  ownership. 

While  it  is  arguable  that  the  JSF’s  O&S  costs  have  been  in  part  “locked  in” 
during  design  (e.g.,  by  the  decision  to  use  stealth  technology,  or  three  variants  of 
one  airframe,  etc.),  it  is  clear  that  the  O&S  costs  are  not  just  unknown,  but  that  there 
is  no  clear  basis  to  either  generate  them  or  to  decide  on  whether  the  degree  of 
variance  between  estimates  have  any  basis  beyond  speculation.  Such  uncertainty  is 
not  particularly  useful  to  the  decision-maker  tasked  with  approving  acquisitions  that 
run  into  many  billions  of  dollars. 

2.  META — DARPA’s  Approach  to  Complexity 

The  fact  that  costs  and  timescales  in  RDT&E  are  uncertain,  and  that 
complexity  underlies  this  uncertainty,  is  currently  being  addressed  by  DARPA  in  its 
META  initiative.  META  aims  to  provide  “novel  methods  for  design  &  verification  of 
complex  systems”  (Eremenko,  2009)  and  to  break  out  from  what  is  seen  as  the 
strictures  imposed  by  systems  engineering  methodologies.  These  strictures  are 
assumed  as  being  likely  causes  of  the  inability  of  defense  acquisition  programs  to 
control  timescales  and  costs  in  RDT&E,  as  they  fail  to  grasp  the  nettle  of  complexity 
as  it  currently  exists. 

While  it  may  appear  that  dealing  with  the  issues  caused  by  complexity  in 
development,  including  reducing  development  costs,  may  have  a  direct  bearing  on 
O&S  costs,  a  number  of  differences  need  to  be  noted.  In  the  findings  reported  in  the 
case  studies  of  this  report,  an  important  driver  of  O&S  costs  originates  in  the 
operating  environment  of  the  system  and  its  interaction  with  the  designed-in  features 
of  the  system.  This  is  a  different  situation  from  the  more  controlled  one  of  RDT&E. 

As  the  case  studies  aim  to  show,  even  the  most  detailed  assumptions  and  plans  for 
the  design,  development,  and  testing  of  a  complex  defense  system  can  miss 
aspects  that  emerge  in  operations  and  support  as  key  drivers  of  costs. 
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In  their  vendor  briefing  for  META,  DARPA  noted  this  fact,  stating  that 
“unmodeled  and  undesired  interactions  lead  to  emergent  behaviors  during 
integration”  (Eremenko,  2009).  This  idea  that  interactions  between  components  in  a 
complex  system  are  important  may  well  be  perfectly  valid  in  the  context  of  RDT&E, 
but  an  assumption  for  the  research  reported  here  is  that  in  the  sphere  of  O&S,  a 
more  important  issue  is  the  interaction  of  the  system  with  its  wider  operational 
environment,  as  well  as  the  wider  commercial  environment  from  which  much  of  the 
support  is  obtained. 

In  META,  DARPA  gives  a  provisional  definition  of  complexity  as  component 
count  for  mechanical  systems,  and  source  lines  of  code  for  software-based  systems. 
This  is,  by  DARPA’s  own  admission,  an  unsatisfactory  definition  of  complexity,  and 
their  work  in  META  seeks  to  develop  new  ones  (Eremenko,  2009).  However,  the 
META  work  is  focused  around  the  interactions  within  a  system.  The  research 
underpinning  this  report  differs  in  that  it  looks  at  the  interactions  both  within  a  system 
and  the  wider  interactions  that  impact  on  that  system  through  the  environment  in 
which  it  is  used  and  supported. 

The  work  reported  here  can  illustrate  some  of  the  differences  between  the 
issues  surrounding  cost  in  operations  and  support,  and  those  in  the  more  commonly 
looked  at  stages  of  research,  development,  test,  and  evaluation.  In  addition,  it  aims 
to  show  how  O&S  relates  to  RDT&E,  both  in  terms  of  differences  and  similarities. 

B.  Research  Approach 

The  research  underlying  this  report  seeks  to  explore  a  complementary 
approach  to  current  analytical  methodologies  in  early  program  stages,  in  order  to 
add  to  the  robustness  of  cost  estimates.  It  aims  to  enable  better  estimates  of  overall 
costs  by  exploring  ways  of  understanding  the  degree  of  possible  cost  variance  from 
the  baseline  provided  by  analytical  techniques. 
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1.  Primary  Literature  and  Research  Questions 


The  original  proposal  that  led  to  this  research  was  based  on  work  looking  at 
the  use  of  academic  and  engineering  literatures  that  explored  the  development  and 
support  of  complex  engineered  systems  and  combat  aircraft  (Pryce,  2009). 

The  two  main  works  that  provide  the  cornerstones  of  the  literature  review  in 
this  section  have  been  selected  as  “primary”  sources,  as  they  represent,  firstly,  the 
leading  synthesis  of  empirical  research  into  the  management  of  civil  complex 
technologies  (Davies  &  Hobday,  2005),  and,  secondly,  the  most  comprehensively 
based,  and  long-term,  analysis  of  the  causes  and  effects  of  design  decisions  on  the 
support  costs  of  military  aircraft  (Reed,  1978). 

In  the  acquisition  of  complex  products  and  systems  (CoPS),  such  as  large 
defense  equipment  programs,  customers  play  a  central  role,  from  design  through  to 
disposal.  In  addition,  it  is  impossible  to  separate  out  the  “artifact”  from  its  process  of 
manufacture,  with  the  two  often  being  designed  in  parallel.  CoPS  are  defined  by 
Davies  and  Hobday  (2005)  as  “any  high-cost,  engineering-intensive  product, 
subsystem,  system,  network,  software  system,  high-technology  service,  capital  good 
or  construct  supplied  by  a  unit  of  production”  (p.  21). 


As  part  of  the  work  undertaken  in  the  CoPS  Innovation  Centre  at  the 
University  of  Sussex  in  the  United  Kingdom,  an  exploration  was  undertaken  of  how 
civilian  firms  that  create  CoPS,  in  fields  such  as  communication  and  transportation, 
move  through  the  value  chain  by  shifting  their  “centre  of  gravity”  (Davies  &  Hobday, 
2005).  This  is  typically  done  to  allow  them  to  modify  their  business  model  to  profit 
from  O&S  activities  and  to  ensure  the  customer  gets  a  better  product  and/or  better 
value  for  money.  The  supply  of  a  service  to  meet  O&S-type  customer  needs,  as  well 
as  the  supply  of  the  original  artifact,  can  constitute  separate  types  of  CoPS  for 
Davies  and  Hobday. 


Implicit  in  this  idea  is  the  ability  of  organizations  undertaking  O&S  for  CoPS  to 
change  the  way  that  the  activities  in  O&S  are  carried  out,  to  reduce  costs  for  a  given 
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capability,  with  support  for  this  coming  from  Gregory  (1989)  and  Hurcombe  (1989). 
This  implication  means  that  in  the  supply  of  civil  CoPS,  it  is  considered  possible  to 
deliver  a  separate,  innovative  solution  to  meet  O&S  needs  that  is  not  limited  by  the 
design  lock-in  characteristics  of  the  artifact.  For  Davies  and  Hobday  (2005),  a  key 
way  to  deliver  such  solutions  is  through  the  ability  to  develop  systems  integration 
skills,  which  underlie  the  ability  of  firms  to  “move  base”  in  the  value  stream. 

This  provides  a  possible  counter  to  the  notion  put  forward  by  Reed  (1978) 
that  O&S  costs  are  effectively  locked  in  by  fundamental  design  decisions  taken  early 
in  a  program.  Reed  suggests  that  this  holds  true  for  all  combat  aircraft,  based  on 
extensive  empirical  case  studies,  and  that  the  opportunities  to  significantly  control 
maintenance  costs  beyond  a  design-determined  limit  are  minimized  by  this.  Reed 
(1978)  states  that  “complex  systems  exhibit  a  constant  failure  rate  throughout  their 
life,”  with  lives  of  20  and  more  years  looked  at.  Reed  shows  that  even  “get  well” 
modifications,  or  the  introduction  of  an  updated  Mark  II  version  of  an  aircraft,  have 
little  effect  on  its  reliability. 

Reed  (1978)  contrasts  this  with  the  previously  (and  subsequently)  assumed 
idea  that  failure  rates  should  form  a  “Bath  Tub”  curve,  with  an  initial  period  of 
increasing  reliability  followed  by  a  level  period  of  system  maturity,  that  is  finally 
followed  by  a  decrease  in  reliability  as  the  system  becomes  old. 

However,  Reed  notes  that  it  is  possible  that  such  a  Bath  Tub  curve  could 
match  the  actual  costs  of  O&S  actions.  However,  he  implies  that  as  these  costs  are 
linked  to  the  failure  rates,  there  is  little  scope  for  controlling  them.  The  increase  in 
costs  over  time,  despite  the  failure  rates  staying  the  same,  appear  to  be  due  to  time- 
dependent  effects  in  the  wider  support  environment,  such  as  the  loss  of  suppliers  of 
component  parts  in  the  supply  chain,  or  of  types  of  maintenance  skills,  or  perhaps 
the  need  for  one  type  of  aircraft  to  carry  all  the  overheads  of  a  component  repair 
chain  after  other  aircraft  types  have  left  service. 
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For  understanding  and  predicting  the  costs  of  O&S  in  complex  defense 
systems,  both  these  works  present  us  with  problems.  The  first  is  that  Davies  and 
Hobday  are  looking  at  CoPS  that  are  far  more  predictable  and  relatively  static  in 
their  use  (telecoms,  construction  and  railways)  compared  to  the  more  dynamic 
nature  of  use  that  many  defense  equipment  programs  face.  They  are  also  looking  at 
CoPS  in  which  the  artifact  and  the  potential  O&S  solution  can  be  readily  separated 
from  each  other.  This  is,  in  part,  a  reflection  of  the  relatively  static  nature  of  the 
CoPS  they  look  at. 

The  second  problem  is  that  Reed  notes  that  the  O&S  lock-in  of  costs  may 
only  apply  to  equipment  in  which  system  repair  is  undertaken  by  the  replacement 
(rather  than  repair)  of  components.  This  is  significant,  as  the  replacement  of 
component  parts  (ignoring  the  time-dependent  effects  noted  previously)  will  provide 
a  fixed  capital  cost,  and  likely  a  nearly  fixed  labor  cost  for  fitting  the  replacement 
part,  whereas  the  possibility  of  repairing  a  component  will  provide  greater  scope  for 
flexibility.  This  is  due  to  the  fact  that  two  different  failures  in  a  single  type  of 
component  can  be  repaired  in  different  ways,  while  replacement  of  the  same 
component  type  is  likely  to  be  the  same  operation.  This  is  especially  true  in  the  case 
of  software,  in  which  all  minor  fixes  can  be  seen  as  the  equivalent  of  “patching”  a 
component,  with  costs  likely  to  vary  between  each  fix,  while  wholesale  changes  in 
modular  blocks  of  software  bring  less  flexibility  and  higher,  essentially  fixed  costs.2 

These  two  issues  mean  that  there  is  a  need  to  explore  further  whether  the 
nature  and  type  of  equipment  being  considered  affects  O&S  costs,  in  this  case 
regarding  combat  aircraft,  as  well  as  whether  the  nature  of  O&S  activities  affects  the 
degree  of  cost  lock-in.  For  our  purposes,  we  will  use  the  term  Dynamic  CoPS  to 
mean  a  complex  engineered  system  that  operates  in  a  complex  environment,  with 


2  The  re-certification  of  safety  critical  systems  is  a  large,  fixed  cost  that  is  to  be  avoided  if  possible. 
This  point  was  forcibly  made  by  several  interviewees. 
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neither  its  operating  environment,  the  role  it  performs,  or  the  nature  and  scope  of  its 
support  systems  (including  both  government  and  contractors)  remaining  constant. 

The  research  presented  in  this  report  can  be  seen  as  answering  the  following 
three  questions: 

■  Is  it  the  case  that  Dynamic  CoPS,  such  as  combat  aircraft,  with  major 
issues  around  operations  in  many  changing  situations,  with  variable 
levels  of  use/damage,  over  many  years,  cannot  be  predictable  enough 
in  use  to  benefit  from  different  solutions  to  their  O&S  needs? 

■  Is  it  the  case  that  by  exploring  the  way  Dynamic  CoPS  are  used  and 
supported,  that  assumptions  that  we  are  constrained  by  design  lock-in 
of  costs  can  be  modified? 

■  How  can  we  design  new  equipment,  or  modify  old  equipment,  to 
benefit  from  such  an  approach? 

2.  Research  Design 

The  research  design  to  explore  these  questions  is  one  of  a  set  of  detailed 
case  studies,  based  on  the  following: 

■  the  U.S./UK  Harrier  combat  aircraft.  This  aircraft  currently  serves  with 
the  United  States  Marine  Corps,  and  served  with  the  UK  Royal  Air 
Force  and  Royal  Navy  until  the  end  of  2010; 

■  the  U.S.  F/A-18  Hornet,  primarily  in  its  first  generation  (A-D 
designation)  variants,  with  some  data  from  the  later  E/F  Super  Hornet 
also  used;  and 

■  the  UK  Tornado  strike  (GR  designation)  aircraft,  used  as  a  control. 

The  main  comparisons  in  this  report  are  between  UK  and  U.S.  Harrier  costs, 
as  these  are  the  most  readily  comparable,  and  as  they  are  the  cases  for  which  the 
most  detailed  data  and  greatest  number  of  interviewees  were  available. 

The  U.S.  F/A-18  program  and  the  UK  Tornado  are  also  featured.  The  data 
used  has  been  made  available  mainly  by  UK  sources,  although  the  numbers  have 
been  made  comparable  to  data  from  the  U.S.  Navy  Visibility  and  Management  of 
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Operating  and  Support  Costs  (VAMOSC)  database3  and  the  Operating  and  Support 
Cost  Analysis  Model  (OSCAM)  cost  modeling  tool.4  This  work  was  further  extended 
by  using  U.S.  originated  data,  also  sourced  from  VAMOSC,  reported  by  other 
researchers  in  the  costing  and  O&S  fields,  such  as  Raman  et  al.  (2003)  and  Price 
and  Coolahan  (2011),  to  assist  in  the  findings. 

An  understanding  of  the  VAMOSC  database  structure,  its  uses,  and  possible 
failings  was  also  obtained  in  order  to  ensure  that  the  data  was  used  correctly  and 
that  its  original  context  was  understood,  as  well  as  to  relate  it  to  the  wider  framework 
of  U.S.  Defense  costing  under  DoDD  5000.4  (DEPSECDEF,  2006). 

3.  Interviews  and  Other  Data  Sources 

A  number  of  anonymous5  interviews  were  carried  out  as  part  of  the  research 
for  this  report.  These  were  loosely  structured,  and  all  interviewees  were  free  to 
terminate  the  interview  when  they  wished. 

In  all  cases  before  interview,  a  copy  of  the  research  proposal  that  led  to  this 
project  was  provided  to  prospective  interviewees.  In  addition,  e-mail  correspondence 
before  the  interview  gave  an  idea  of  the  types  of  questions  to  be  asked. 

During  interviews  the  author  allowed  participants  to  talk  freely  about  issues 
that  they  thought  mattered,  both  during  and  after  their  answers  to  prepared 
questions.  All  interviewees  agreed  to  allow  follow-up  questions  to  be  e-mailed,  and 
in  several  cases  secondary  telephone  interviews  were  carried  out. 


3  Notably  the  “old”  VAMOSC  system,  before  restructuring  of  the  database  in  1999. 

4  The  understanding  of  OSCAM  was  greatly  helped  by  the  “walk  through”  provided  by  Dale  Sherman 
of  the  Society  for  Cost  Analysis  and  Forecasting. 

5  A  number  of  interviewees  were  happy  to  be  named,  while  others  wished  to  remain  anonymous.  In 
order  to  give  equivalent  weight  to  the  views  of  all  interviewees,  anonymity  has  been  preserved  for  all. 
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In  addition,  informal  discussions  with  contacts  made  during  the  research  led 
to  correspondence  with  contacts  that  were  not  available  for  a  formal  interview  but 
were  happy  to  provide  written  information  to  be  used  in  the  research. 

Additional  sources  of  formal  data  were  obtained  from  BAE  Systems  and  via 
the  Society  for  Cost  Analysis  and  Forecasting.  In  all  cases,  the  people  providing  the 
data  were  available  to  explain  the  sources  and  limitations  of  it,  as  well  as  to  make 
clear  the  limits  of  its  use. 
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Section  II.  Case  Studies 


The  case  studies  provide  the  main  empirical  content  of  this  report.  They  are 
based  on  a  number  of  combat  aircraft  programs  and  are  intended  to  show  the  nature 
of  the  issues  that  emerge  in  operations  and  support.  They  are  also  intended  to  show 
how  these  issues,  seen  as  interactions,  can  explain  the  causes  of  cost  variance  in 
O&S,  as  well  as  suggesting  possible  ways  of  using  this  knowledge  to  develop  better 
costing  tools. 

The  cases  could  also  serve  to  highlight,  in  a  form  of  “backcasting,”  how  the 
approaches  put  forward  in  Section  III  of  this  report  could  have  been  utilized  in  the 
JSF  program.  It  is  telling  that  the  data  used  in  these  cases  were  mostly  available  at 
the  early  stages  of  the  JSF’s  evolution  in  the  mid-1990s. 

The  main  idea  explored  in  the  cases  is  that  the  degree  of  complexity  in  a 
project  may  have  a  significant  effect  on  the  range  of  variance  in  its  possible  O&S 
costs.  An  initial  assumption,  to  be  tested  using  the  cases,  is  that  the  greater  the 
degree  of  complexity,  the  narrower  the  room  for  maneuver  in  reducing  O&S  costs. 
Essentially,  the  idea  tested  is  that  greater  complexity  brings  greater  cost  lock-in, 
similar  to  that  expounded  by  Reed  (1978).  Figure  1  shows  an  overview  of  the  case 
studies  to  be  looked  at. 


- > 

Increasing  system  complexity 


Primary  comparison  Secondary  comparison 


Figure  1.  Aircraft  Program  Comparison  Framework 
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The  cases  explore  the  following  aspects  of  O&S: 


1 .  the  degrees  of  variance  in  O&S  requirements  between  Hamers  in  the 
UK  and  U.S.  and  other  aircraft  (F/A-18  and  Tornado),  to  establish  how 
the  degree  of  “designed-in”  complexity,  patterns  of  operational  use, 
and  so  forth,  may  vary;  and 

2.  the  UK’s  Harrier  GR.9  upgrade  and  the  U.S.’s  AV-8B  OSCAR  upgrade, 
to  explore  how  design  lock-in  issues  were  tackled  in  system  updates 
never  imagined  by  their  original  designers  or  users. 

It  should  be  noted  that  it  is  an  assumption  in  this  report  that  factors  such  as 
arisings  and  operational  effects,  discussed  in  the  next  section,  have  a  rough 
equivalence  in  cost  terms  across  all  users.  This  is  assumed  to  mean  that  they  result 
in  rectification  actions  which  lead  to  maintenance  man-hours  that  are  charged  at 
nationally  equivalent  rates,  as  well  as  the  consumption  of  spare  parts  that  have 
similar  costs.  On  this  basis,  the  factors  explored  are  taken  to  be  good  proxies  for 
actual  costs  incurred  over  time. 

A.  Case  Study  1 :  Aircraft  O&S,  Design,  and  Use 

The  approach  to  estimating  the  degree  of  complexity  put  forward  in  this 
research  is  based  on  the  idea  that  it  is  not  component  count  or  source  lines  of  code 
that  matter,  but  rather  the  total  number  of  interactions  between  engineered 
components,  the  way  an  aircraft  is  used,  and  the  organizations  undertaking  the  O&S 
activities  on  the  aircraft.  The  latter  two  areas  are  referred  to  as  the  environment,  that 
is,  all  that  is  external  to  the  system  itself. 

The  assumption  is  that  the  overall  effect  of  these  interactions,  whether  they 
originate  from  designed-in  features  or  from  the  wider  environment,  would  be 
revealed  by  comparisons  between  the  number  and  type  of  defects  experienced 
during  service  use  of  the  aircraft  and  the  related  failures  these  defects  produced  on 
the  aircraft. 

The  defects  are  here  called  arisings,  and  the  failures  operational  effects,  in 
line  with  UK  O&S  language.  An  aircraft  can  still  continue  to  fly  a  mission  with  an 
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arising,  but  an  operational  effect  will  mean  that  a  mission  cannot  continue  as 
planned.  An  important  aspect  is  that  the  link  between  arisings  and  operational 
effects  is  not  necessarily  a  technical  one,  but  subject  to  operational  use  and 
maintenance  patterns.  The  purpose  of  this  detailed  examination  of  these  figures  is  to 
see  how  such  use  patterns  interact  with  the  aircraft  to  produce  an  overall  pattern  of 
O&S  costs. 

1.  O&S  Data  Analysis 

Figure  2  provides  an  overview  of  the  level  of  arisings  and  operational  effects 
on  the  aircraft  platforms  that  form  the  basis  of  the  case  studies.  The  data  presented 
are  relatively  old  (mid-1980s),  but  have  the  great  value  of  being  for  a  similar  period 
of  use  for  each  platform  and  having  been  cross  checked  from  a  number  of  data 
sources  that  have  additional  contextual  explanations  for  their  sources.  Finding  data 
that  are  comparative  on  such  a  basis  is  essential  to  allow  meaningful  comparisons  to 
be  made. 


Type 

Arisings 

Op  Effects 

RAF  Harrier  1  <A) 

2564 

61.9 

RN  Harrier  1  w 

1449 

51.9 

Tornado  (B) 

2122 

140.0 

AV-8B  CA) 

1096-1330 

24.1-29.8 

F/A-18A/B  (B) 

1265 

33.5 

Sortie  length  effects: 


Increasing  sortie  duration 
by  factor ‘t’  increases 
occurrences  by  function  Vt 
and  decreases  rates  per 
flying  hour  by  the  ratio  1/  Vt 


Notes:  Some  AV-8A/C <fl)  and  UK/US  Phantom <B)  data  used  for  comparison  Sources:  BAES/VAMOSC 


Figure  2.  Aircraft  Reliability  and  Failure  Rates 

Note.  Numbers  are  per  1 ,000  flying  hours. 

Three  main  points  should  be  noted  in  relation  to  the  data  in  Figure  2.  First,  the 
selection  of  three  variants  of  the  Harrier  family,  from  two  generations  used  by  the 
Royal  Air  Force  (RAF),  Royal  Navy,  and  United  States  Marine  Corps  (USMC), 
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allows  the  effects  of  issues  such  as  different  levels  of  technology,  operational  use 
patterns,  and  so  forth,  to  be  compared.  These  can  be  considered  as  forming  much 
of  the  environment,  external  to  the  technical  features  of  the  aircraft. 

Second,  for  the  AV-8B,  F/A-18A/B,  and  Tornado,  the  data  presented  is  for 
early  production  batches  during  a  period  in  which  they  were  still  being  introduced 
into  service.  This  may  mean  that  there  was  an  improvement  in  reliability  in 
subsequent  years,  although  discussions  with  one  interviewee,  who  used  this  data  in 
support  of  Harrier  aircraft,  revealed  their  belief  that  for  all  but  the  Tornado,  the  data 
were  robust.  The  Tornado  data  was  complicated  by  the  persistent  unreliability  of  the 
type  during  the  1980s  and  the  changes  introduced  in  a  major  “get  well”  program. 
Cross  checking  with  figures  given  by  Hurcombe  (1989)  indicates  that  the  Harrier  I 
and  AV-8B  data  would  be  accurate,  and  gives  an  explanation  for  the  Tornado  data 
that  would  indicate  it  was  not  inaccurate. 

Third,  and  of  great  significance  for  this  research,  was  the  difficulty  in 
comparing  U.S.  and  UK  data  which  use  different  “accounting”  practices.  Access  to 
the  old  VAMOSC  database,  and  cross  checking  with  the  work  of  Raman  et  al. 
(2003),  indicates  that  much  of  this  difficulty  can  be  avoided  by  the  use  of  the  old 
VAMOSC  data,  which  has  been  used  here. 

Some  of  the  comparisons  between  the  U.S.  and  UK  Harriers  were  made 
possible  by  some  data  for  the  USMC’s  own  first  generation  Harriers.  However, 
conversion  of  U.S.  data  to  UK  formats  does  mean  that  accounting  allocations  need 
to  be  made  that  may  be  slightly  wrong,  hence  the  spread  of  data  points  for  the  AV- 
8B  and  the  F/A-18A/B. 

The  comparison  between  the  three  Harrier  variants  in  Figure  2  illustrates  a 
number  of  issues.  RAF  Harrier  sorties  were  of  lower  duration  than  Royal  Navy  ones, 
as  well  as  being  more  punishing  on  the  airframe  as  they  were  flown  at  lower 
altitudes.  The  Harrier  is  well  known  for  subjecting  much  of  its  avionics  and  airframe 
systems  to  a  punishing  acoustic,  thermal,  and  vibration  environment,  which  is  the 
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cause  of  many  system  failures,  and  was  not  amenable  to  prediction  using  standard 
methods/test  spectra,  and  so  forth  (see  Beier,  1987).  Flight  at  low  level  and  high 
throttle  settings  exacerbate  these  problems.  On  the  other  hand,  naval  operations 
introduce  issues  of  corrosion  not  present  on  land,  although  the  youth  of  the  Sea 
Harrier  fleet  for  the  period  under  study  happily  makes  this  relatively  unimportant  for 
the  period  covered  by  these  data. 

The  box  on  the  right  of  Figure  2  illustrates  that  these  differences  can  be 
simplified  into  a  general  statement  on  the  effect  of  sortie  lengths  on  the  occurrence 
rates  for  arisings  and  operational  effects,  at  least  for  aircraft  of  a  similar  technology 
level.  These  sortie  lengths  are  a  product  of  the  operational  pattern  experienced  by 
the  aircraft,  and  of  their  different  roles,  with  the  Harrier  I  used  for  ground  attack,  and 
the  Sea  Harrier  primarily  as  an  air  combat  fighter. 

The  Royal  Navy  Sea  Harriers  were  of  a  similar  technology  level  to  the  RAF 
aircraft,  although  built  five  to  ten  years  later,  with  more  modern  avionics  and  some 
system  improvements  incorporated.  The  AV-8B  Harriers  of  the  USMC  shown  in 
Figure  2  were  of  a  new  generation  design,  incorporating  a  new  wing  made  of  carbon 
fiber,  new  avionics,  and  some  revised  airframe  systems.  However,  the  retention  of 
major  parts  of  the  airframe  (manufactured  in  the  UK),  most  significantly  the  rear 
fuselage  that  experiences  the  harshest  loadings  in  the  Harrier  ( and  the  systems — 
hydraulics,  electrics,  conditioning — contained  within  it),  as  well  as  the  undercarriage 
units,  that  were  all  derived  from  the  first  generation  Harriers,  allows  a  good  basis  for 
comparison  between  the  three  types  of  Harrier. 


The  data  shows  that  the  AV-8B  Harriers  had  similar,  if  slightly  lower,  arising 
rates  to  the  Sea  Harrier,  but  much  lower  operational  effects  rates.  In  part  this  was 
due  to  environmental  factors,  such  as  weather.  Put  simply,  the  weather  in  Yuma, 
Arizona,  is  much  drier  than  at  Yeovilton  in  the  UK,  while  operations  from  ships  in  the 
North  Atlantic,  as  well  as  on  combat  operations  in  the  South  Atlantic,  had  an  adverse 
effect  on  Sea  Harrier  rates.  This  is  due  to  moisture  causing  electrical  failures,  rather 
than  any  corrosion.  The  greater  fuel  capacity  and  more  efficient  wing  for  cruising 
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flight  of  the  AV-8B  allowed  longer  sorties  than  those  of  the  Sea  Harrier,  helping  to 
give  a  favorable  operational  effects  figure. 

Although  the  main  data  presented  in  Figure  2  relate  to  comparisons  between 
the  Harrier  family,  data  are  also  provided  for  the  more  complex  F/A-18A/B  and  the 
British  PANAVIA  Tornado  GR.1 .  In  the  case  of  these  aircraft,  it  was  thought  that  the 
major  design  differences  would  make  comparison  more  difficult.  However,  there  was 
some  hope  that  the  fact  that  they  are  both  twin-engined  types,  and  that  the 
complexity  of  the  “swing  wing”  on  the  Tornado  may  have  some  equivalent  in  the 
added  complexity  of  the  “navalization”  features  for  the  F/A-18  Hornet. 

However,  as  Figure  2  shows,  it  is  apparent  that  the  differences  in  the  arisings 
and  operational  effects  figures  were  very  significant.  This  is  explicable  in  part  due  to 
factors  mentioned  in  relation  to  the  Harrier  data — different  mission  profiles,  different 
environmental  effects,  and  so  forth — but  the  data  appear  to  reveal  the  fact  that  the 
F/A-18A/B  was  inherently  more  reliable  by  design  than  the  Tornado.  An  attempt  at 
controlling  UK/U.S.  accounting  differences  using  old  F-4  Phantom  data  did  not 
provide  any  greater  insight.  Additional  data  showed  that  later  batches  of  Tornado 
were  significantly  more  reliable,  as  a  result  of  the  get  well  program  reported  by 
Hurcombe  (1989). 

This  program  was  very  costly,  with  the  equivalent  of  more  than  100%  of  the 
basic  Tornado  component  count  being  replaced.  Indications  from  this  data,  as  well 
as  from  interviews  undertaken  with  experienced  aircrew  now  working  in  industry,  are 
that  this  is  in  part  explicable  due  to  the  RAF  failing  to  support  the  Tornado  using  the 
maintenance  strategy  for  which  it  was  designed,  but  that  during  operational 
deployments  (notably  to  warm,  dry  climates),  the  correct  servicing  and  spares 
approach  was  followed,  significantly  improving  reliability. 

It  is  interesting  to  note  the  relative  similarities  between  AV-8B  and  F/A-18 
data  in  Figure  2 — both  aircraft  originating  at  the  same  time,  from  the  same  design 
team,  and  sharing  some  systems.  Analysis  of  these  similarities  and  their  causes  has 
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shown  that  a  significant  factor  behind  this  lay  in  the  use  of  improved  manufacturing 
techniques  for  the  Harrier  and  Hornet  at  what  is  now  Boeing,  St.  Louis.  Cross 
checking  with  F-4  Phantom  II  data  appeared  to  back  this  up.  This  was  further 
reinforced  by  an  interviewee  who  was  familiar  with  first  and  second  generation 
Harriers  and  Hornets. 

In  short,  the  use  of  more  accurate  assembly  techniques,  derived  in  part  from 
computer  aided  design  and  computer  aided  manufacture,  allowed  for  improved 
repeatability  in  the  installation  of  equipment.  This  meant  that  a  smaller  number  of 
mechanical  interactions  between  poorly  secured  components  emerged  in  service 
than  on  previous  aircraft  types. 

This  data  illustrates  the  idea  that  interactions  are  not  simply  about  the  number 
of  components  parts,  but  rather  are  caused  by  a  range  of  factors.  It  can  certainly  be 
said  that  the  number  of  components,  and  the  number  of  their  interactions,  is  not  the 
proper  measure  of  complexity.  The  number  of  components  in  the  Harrier  variants 
was  not  greatly  different  between  them,  but  the  data  shown  in  Figure  2  reveals 
widely  varying  arising  and  operational  effect  rates.  These  differences  come  about 
through  the  effect  of  interactions  both  within  the  system  and  without  it,  in  the  wider 
environment  of  manufacture,  use,  and  support.  Factors  such  as  sortie  rates, 
operational  flight  profiles,  the  nature  of  maintenance  patterns,  manufacturing 
technologies,  and  so  forth,  are  the  sources  of  the  total  set  of  interactions  that  the 
aircraft  components  endure. 

2.  Undercarriage  Example  of  Detailed  O&S  Matters 

To  understand  the  factors  that  affect  O&S  more  deeply,  an  example  of  a  part 
of  the  aircraft  that  was  largely  common  to  all  three  variants  of  the  Harrier  was 
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required.  The  main  undercarriage  (landing  gear)  units  were  selected.6  Data  for  the 
share  of  overall  O&S  LCC  costs  of  the  RAF  Harrier  /’ s  undercarriage  are  shown  in 
Figure  3.  It  can  be  seen  that  the  undercarriage’s  share  of  the  LCC  O&S  costs  are 
largely  typical  of  other  major  systems,  that  is,  it  is  not  unusual  in  its  percentage  of 
overall  costs.  This  was  seen  as  making  the  undercarriage  a  good  candidate  to 
explore  in  further  detail. 


Figure  3.  RAF  Harrier  I  LCC  O&S  Costs 

Undercarriage  units  of  combat  aircraft  are  high  value  items  that  are  designed 
to  meet  an  operating  life  according  to  a  certain  assumed  spectrum  of  use.  They  are 
built  to  last,  and  are  safety  critical  as  their  failure  during  take-off  or  landing  can  lead 
to  total  loss  of  the  aircraft.  Undercarriage  units  are  exposed  to  heavy  stresses 
throughout  their  lives,  and  are  subject  to  widely  varying  loads  depending  on  the 
overall  pattern  of  use  of  the  aircraft  to  which  they  are  fitted. 


6  The  Harrier  has  an  unusual  “bicycle”  main  undercarriage  unit,  with  wingtip  outriggers  on  the  RAF 
Harrier  I/Sea  Harrier  and  mid-wing  outriggers  of  different  design  on  the  AV-8B.  However,  the  main 
units  have  only  minor  differences,  for  example,  some  strengthening  and  lash-down  lugs  for  ship- 
borne  use. 
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These  factors  can  lead  to  a  heavy,  but  variable,  maintenance  burden,  with 
frequent  inspections  required,  and  repair  or  replacement  of  damaged  units  often 
required.  For  naval  aircraft,  or  short  take-off  and  vertical  landing  (STOVL)  aircraft 
such  as  the  Harrier,  there  are  many  additional  sources  of  fatigue  and  other  damage 
to  the  undercarriage,  compared  to  land-based  aircraft. 

One  key  difference  between  UK  and  U.S.  undercarriage  O&S  is  that 
maintenance  of  such  units  is  a  more  specialized  trade  in  the  U.S.,  to  the  extent  of 
personnel  specializing  down  to  the  level  of  main  or  nose  gear  support.  In  the  UK, 
maintenance  is  based  on  a  more  general  trade  structure,  but  that  may  also  mean 
that  all  units  are  inspected  on  an  equal  basis  of  periodicity  and  degree. 

Operations  on  the  Harrier  have  led  to  constant  discoveries  of  undercarriage 
O&S  issues  that  needed  to  be  addressed.  Although  the  main  undercarriage  was 
very  robust,  being  designed  to  operate  off  base  and  to  take  many  unusual  loads, 
such  as  landing  while  flying  backwards,  these  discoveries  were  near  impossible  to 
predict  and  meant  that  the  real-world  experience  of  the  undercarriage  in  use  differed 
from  the  original  design  spectrum  that  they  were  built  to  meet. 

For  example,  as  Burton  (1996)  reports,  seemingly  minor  differences  in  the 
build  quality  of  the  ski-jump  ramps  of  the  UK’s  Invincible  Class  light  aircraft  carriers 
seriously  affected  the  life  of  the  undercarriage  units,  depending  on  which  ship  was 
being  operated  from.  These  ship  build  quality  differences  were  not  part  of  the 
original  design  assumptions,  or  subsequent  modeling  undertaken  for  a  new  ski-jump 
design  fitted  to  UK  aircraft  carriers  and  its  effect  on  the  aircraft’s  operating  limits,  and 
led  to  unexpected,  and  unexplained,  cracking  in  the  undercarriage  units. 

Upon  investigation,  down  to  individual  aircraft  and  mission  levels,  it  was 
discovered  that  the  undercarriage  damage  suffered  was  not  particular  to  the  role  or 
mission  profile  of  the  aircraft,  or  to  the  type  of  Harrier,  but  to  the  particular  ship  of  a 
class  that  they  were  operating  from.  The  damage  was  expensive  to  repair,  but 
absolutely  necessary  in  order  to  avoid  a  catastrophic  failure  mode  that  could  not  be 
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predicted.  Such  a  failure  would  lead  to  loss  of  an  aircraft  and  likely  serious  damage 
to  the  ship.  However,  it  was  avoidable.  The  issues  of  the  variability  of  carrier  deck 
design  on  the  class  of  ships  concerned  were  known  to  the  aircraft  design  team  at 
least  a  decade  before,  with  pitting  and  so  forth  causing  problems  on  both  deck  and 
in  the  hangar  (Brooklands  Museum,  1985).  However,  in  the  calculation  of 
undercarriage  loads  carried  out  during  the  design  of  the  Sea  Harrier,  a  smooth  deck 
was  assumed,  based  on  design  rules  created  by  the  UK  Ministry  of  Defence 
(National  Archives,  1978). 

3.  Summary  of  Case  Study  1 

The  undercarriage  example  has  been  given  to  illustrate  the  peculiarities  of 
the  individual  incidents  that  make  up  the  totality  of  the  data  presented  in  Figure  2, 
and  to  give  an  idea  of  how  they  can  emerge  unexpectedly.  However,  the  fact  that 
the  Harrier’s  undercarriage  was  of  a  robust  design  meant  that  there  were  not  any 
failures,  just  arisings  that  were  repairable  (and  are  similarly  so  for  the  AV-8B;  see 
Hullander  and  Walling,  2008).  The  rate  and  nature  of  these  arisings,  however,  were 
not  designed  in,  although  the  original  characteristics  of  the  undercarriage  units  were. 
The  arisings  were  due  to  the  peculiarities  of  the  aircraft’s  use. 

These  types  of  issues  have  emerged  in  a  range  of  other  examples  in  the 
research.  Not  just  the  type  of  operational  flight  profile,  but  even  who  is  flying  the 
aircraft  can  have  an  effect.  As  one  interviewee  ( ex-Tornado  aircrew)  put  it,  “If  the 
same  aircraft  is  flown  by  the  same  people  every  day,  it  doesn’t  break.”  Put  simply, 
the  fewer  times  switches,  ejector  seat  rigging,  and  so  forth,  are  adjusted,  the  fewer 
interactions,  and  the  fewer  failures  occur. 

In  summary,  while  it  is  true  that  the  design  stage  may  well  lock  in  some 
aspects  of  O&S,  and  it  is  true  that  some  parts  are  more  liable  to  break  than  others, 
and  some  are  easier  to  fix  depending  on  how  they  are  designed,  this  is  not  the 
whole  story.  Rather,  the  total  fleet  average  of  arisings  and  operational  effects  are 
caused  by  a  wide  variety  of  interactions  between  systems  on  board  the  aircraft, 
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between  the  aircraft  and  its  wider  operational  and  support  environment,  and  beyond, 
into  the  interactions  of  these  factors  with  changes  in  production  and  design  made 
without  any  ability  to  predict  their  likely  outcomes. 

B.  Case  Study  2:  Harrier  Upgrades 

The  first  case  study  illustrated  how  the  factors  influencing  O&S  are  affected 
by  operational  use  and  other  environmental  factors.  In  this  second  case  study,  we 
will  explore  how  the  interactions  in  contracting  for  O&S,  and  their  link  to  operationally 
urgent  updates,  can  be  key  drivers  of  costs.  We  also  look  at  how  these  costs  can  be 
contained  though  the  use  of  technological  and  managerial  innovations,  such  as  the 
use  of  commercial  off-the-shelf  (COTS)  technology  insertion. 

The  second  case  explores  the  update  of  the  mission  computer  on  the  UK’s 
Harrier  GR. 9  program,  undertaken  by  BAE  Systems,  and  also  looks  at  the 
comparable  Open  Systems  Core  Avionics  Requirement  (OSCAR)  update  for  the 
U.S.  Marine  Corps’  AV-8B.  As  Roark,  Devers,  Myers,  Suchan,  and  Walt  (2008)  have 
noted,  it  is  harder  to  have  visibility  of  costs  when  O&S  is  implemented  by  a 
contractor,  a  problem  which  is  apparently  beginning  to  undermine  the  utility  of 
VAMOSC-type  data.  This  means  that  to  understand  how  contractors  undertake  such 
activities  will  be  valuable  to  understanding  the  causes  of  O&S  costs  that  may  not  be 
captured  in  the  data  used  in  current,  analytical  cost  estimation  techniques,  such  as 
those  recommended  by  DoDD  5000.4  (DEPSECDEF,  2006). 

1.  Harrier  GR.9 

The  Royal  Air  Force's  Harrier  GR.9  was  the  last  version  of  the  Harrier  II  family 
introduced  in  the  UK.  As  such  it  is  similar  to  the  U.S.  Marine  Corps’  AV-8B  in  basic 
structure,  systems,  and  powerplant,  but  differs  in  its  fit  of  mission  avionics.  To 
update  the  previous  RAF  version  of  the  Harrier  II  (the  GR.7)  to  become  the  GR.9,  a 
mission  systems  update,  termed  the  Harrier  Integrated  Weapons  Programme  (IWP), 
was  devised  to  bring  together  a  number  of  discrete  weapon-system  enhancement 
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projects.  The  IWP  formed  the  basis  of  the  GR.9  and  T.12  aircraft,  the  latter  a  two- 
seat  training  version. 

A  principal  feature  of  the  GR.9  upgrade  was  a  state-of-the-art  MIL-STD-1760 
Stores  Management  System  (SMS),  which,  combined  with  the  new  High  Order 
Language  (Ada)  Operational  Flight  Programme  (OFP)  software  and  a  new  Open 
System  Mission  Computer  (OSMC),  permitted  the  aircraft  to  interact  with  new 
weapons  and  sensors. 

In  April  2002,  BAE  Systems  received  an  interim  contract  for  the  development 
of  the  full  GR.9  aircraft.  A  further  £150  million  contract  was  signed  in  January  2003 
for  non-recurring  work,  mainly  software  development  and  flight  test.  The  first  aircraft 
flew  in  May  2003,  with  an  initial  batch  of  aircraft  completed  by  the  end  of  2003. 
Operational  release  occurred  in  September  2006.  The  full  modification  program  had 
a  value  of  £500  million,  including  support  costs.  The  update  program  was  managed 
through  the  Future  Integrated  Support  Team  (FIST),  a  joint  industry/UK  Ministry  of 
Defence  (MoD)  initiative,  with  engineering  design  undertaken  at  BAE  Systems 
Farnborough,  and  development  and  flight  test  based  at  BAE  Systems’  Warton  site. 
The  scope  of  the  Harrier  GR.9  upgrade  work  managed  by  FIST  involved  the 

baseline  recovery,  re-design,  and  re-implementation  of  significant 
aspects  of  the  avionic  system,  and  associated  subsystem  design; 

procurement,  integration,  and  test; 

a  complete  recovery  and  rewrite  of  the  software  for  the  central 
computer  controlling  the  avionic  and  weapon  systems  (some  250,000 
lines  of  code); 

rewiring  of  the  aircraft  (over  five  miles  of  wiring  per  aircraft  was 
removed  or  replaced); 

the  selection  and  integrated  management  of  major  international 
vendors  through  competitive  tender; 

structural  and  aerodynamic  clearances; 
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following: 

1. 

2. 

3. 

4. 

5. 

6. 


7.  the  management  of  five  instrumented  development  Hamers  to  provide 
test  clearance  and  certification  of  each  capability;  and 

8.  the  manufacturing  of  parts/equipment  and  their  embodiment  to 
upgrade  to  GR.9  standard  across  the  Harrier  fleet.  (Interviews,  Pryce, 
2009) 

The  overall  program  was  implemented  in  a  new  way.  Traditionally  in  UK 
service,  first  line  (e.g.,  preparation  for  flight)  and  second  line  (e.g.,  hangar  based 
checks)  maintenance  had  usually  taken  place  using  military  personnel  on  the  main 
operating  base  (or  aircraft  carrier),  with  major  updates,  repair,  and  overhaul  (third 
and  fourth  line  maintenance)  either  taking  place  at  a  specialized,  Ministry  of  Defence 
owned  and  operated  facility  separate  from  the  main  operating  base,  or  in  a  return  to 
works  program  where  aircraft  are  returned  to  the  manufacturer/support  company  for 
work  to  be  carried  out. 

However,  the  modifications  for  the  Harrier  GR.9  update  were  embodied  using 
a  novel  organizational  structure.  The  Joint  Upgrade  and  Maintenance  Programme 
(JUMP)  involved  both  scheduled  maintenance  and  upgrade  activities  centered  on  a 
single  main  base  location,  with  industry  and  military  personnel  co-operating  in  so 
called  depth  servicing  (combining  some  second  and  all  third  and  fourth  line  work). 
This  was  carried  out  using  a  pulse  line,  involving  concepts  adapted  from  lean 
manufacturing,  to  speed  up  the  work  and  reduce  costs. 

It  was  therefore  a  very  complex  program,  involving  many  participants  in 
industry,  government,  the  RAF,  and  the  Royal  Navy  (who  operated  the  GR.9  after 
their  own  dedicated  Sea  Harrier  fleet  was  retired  in  2006).  Matters  were  further 
complicated  by  the  need  to  incorporate  unplanned  Rapid  Technology  Insertion  (RTI) 
activities  as  a  result  of  ongoing  UK  Harrier  operations  in  Afghanistan  that  emerged 
after  the  program  was  planned.  These  tested  the  ability  of  the  technical  systems  and 
organizations  involved  in  the  update  effort  to  adjust  to  new  requirements  while  still 
implementing  the  existing  plan. 
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At  the  heart  of  the  Harrier  GR.9  update  was  the  use  of  a  COTS  mission 
computer  system,  provided  by  GE  Aviation  (formerly  Smiths  Aerospace).  This 
shared  a  common  chassis  and  some  cards  with  the  OSCAR  mission  computer  that 
was  used  by  Boeing  to  update  the  USMC’s  fleet  of  AV-8B  Harriers  (see  the 
discussion  on  this  later  in  this  section),  but  development  was  handled  by  the  UK 
FIST  team  as  if  this  was  a  new  project.  Key  to  the  program’s  successful  execution 
was  the  fleet  of  development  test  aircraft.  With  all  Harrier  aircraft,  there  are 
additional  issues  that  exacerbate  the  testing  cycle  for  modifications  and  new 
variants.  Vibration  levels  for  Harrier  avionics  are  not  based  on  a  fixed  standard  to 
which  a  system  can  necessarily  be  certificated  before  use  on  the  aircraft  (Interview, 
Beier,  1987),  unlike  other  aircraft,  which  can  use  such  standards.  Due  to  the  intense, 
engine  induced  vibration  experienced  in  the  rear  fuselage  of  the  Harrier,  special 
certification  of  aircraft  systems  are  required.  These  can  possibly  extend  the  testing 
cycle  of  upgrades  and/or  limit  COTS  insertion.  Of  particular  importance  is  that  when 
safety  critical  systems  are  being  updated,  COTS  “can  rapidly  lose  their  attraction” 
(according  to  a  BAE  Harrier  interviewee). 

As  with  the  example  of  the  Harrier  undercarriage  given  in  Case  Study  1,  the 
mission  computer  is  a  safety  critical  item.  This,  in  part,  explains  why  the  testing  cycle 
can  be  long — it  is  necessary  to  ensure  that  the  safety  of  the  system  has  been 
proven,  and  analytical  models  or  bench  testing  are  not  adequate  to  do  this. 

However,  the  need  to  incorporate  pre-planned,  incremental  capability  levels  to  the 
mission  computer  operational  flight  program  (OFP) — as  well  as  the  need  to  change 
OFP  software  in  response  to  emerging  RTI  needs  in  light  of  urgent  operational 
requirements  emerging  from  operations  in  Afghanistan — meant  that  a  stable, 
relatively  slow  approach  to  the  testing  cycle  was  not  possible. 

In  order  to  get  the  required  results  in  the  shortest  possible  time,  BAE 
Systems’  Harrier  GR.9  team  decided  to  use  a  number  of  shortcuts  in  developing  the 
safety  case  of  the  mission  computer.  These  consisted  of  both  simple  tools  and 
methods  of  working  that  allowed  clear  communication  to  all  participants  in  the 
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company,  its  suppliers,  and  customers  in  the  RAF  and  Royal  Navy  (Lucas,  2008). 
This  considerably  speeded  up  the  insertion  of  new  capabilities  and  technologies, 
and  also  made  visible  to  all  stakeholders  what  was  being  done,  and  what  needed  to 
be  done  in  the  future. 

Central  to  the  ability  to  do  this  was  BAE  Systems’  control  of  the  OFP.  Rather 
than  control  of  the  OFP  software  residing  in  the  supplier  of  the  computer  itself,  or  in 
the  customer’s  O&S  organization,  BAE  ensured  that  they  had  full  control  over  the 
software  upgrade.  This  allowed  them  to  use  the  tools  and  development  environment 
with  which  they  were  familiar.  The  OFP  was  particular  to  the  Harrier  GR.9,  not  being 
based  on  a  modular  re-use  of  software.  In  addition,  GR.9  COTS  software  languages 
such  as  C++,  as  used  on  the  OSCAR  program,  were  used  less  than  the  older  Ada 
language,  which  had  a  well  understood  development  environment.  As  the  OFP  was 
frequently  updated,  such  control  allowed  BAE  systems  to  speed  up  the  overall 
development  process  in  response  to  the  needs  from  Afghanistan. 

In  order  to  meet  the  needs  of  the  operational  force  in  Afghanistan,  BAE 
Systems  carried  out  a  number  of  privately  funded  RTI  programs.  One  of  these  was 
to  add  a  new  targeting  pod  capability  onto  the  Harrier.  It  was  found  that  the 
enhanced  target  identification  and  weapon  delivery  capability  desired  could  be 
realized  with  the  integration  of  the  Sniper  Pod,  provided  by  Lockheed  Martin,  and 
called  the  Advanced  Targeting  Pod  (ATP)  when  used  on  the  Harrier. 

A  meeting  between  BAE  Systems  and  Lockheed  Martin  in  early  September 
2006  confirmed  the  feasibility  of  an  RTI  program.  In  a  compressed  program  of 
hardware  and  software  development,  aircraft  integration,  rig  testing,  and  certification, 
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the  team  conducted  the  first  flight  just  two  months  after  the  first  RTI  integration 
meeting.7 

BAE  Systems  further  developed  the  ATP  solution  and  gave  interim  clearance 
advice  to  enable  the  Operational  Evaluation  Unit  to  conduct  trials  in  less  than  four 
weeks  from  receipt  of  the  Request  for  Quotation. 

Allied  to  the  delivery  of  the  ATP,  the  Harrier  fleet  was  also  fitted  with  new 
laser  guided  (Paveway  IV)  bombs.  The  integration  of  these  followed  the  same  type 
of  accelerated,  RTI  program.  The  data  provided  by  one  interviewee  shows  that  this 
approach  reduced  timescales  on  this  RTI  exercise  from  18  months  down  to  three 
months,  with  the  cost  of  the  integration  effort  reduced  from  £15  million  to  £5  million. 

It  is  clear  that  this  approach  can  work,  and  deliver  greatly  reduced  O&S  costs  on 
system  update  activities. 

This  is  an  interesting  case,  as  the  basic  ATP  system  is  identical  to  the  internal 
targeting  system  to  be  used  on  the  Joint  Strike  Fighter,  which  will  serve  from  UK 
aircraft  carriers.8  Current  costing  of  the  trials  and  evaluation  of  the  JSF  in  the  UK  are 
based  on  clearance  for  the  internal  targeting  system  on  board  the  UK’s  new  aircraft 
carriers  using  a  far  slower  testing  cycle  than  was  shown  to  be  possible  in  the  Harrier 
ATP  exercise.  Interviewees  from  the  BAE  Harrier  team  were  quite  clear  in  their  belief 
that  translation  of  the  Harrier  development  environment  into  the  JSF’s  testing  of  the 
targeting  system  was  both  possible  and  would  be  much  cheaper,  quicker,  and  less 
risky  than  current  plans.  By  understanding  the  development  environment  used  by 
the  Harrier  team,  the  JSF  program  could  reduce  costs  in  both  testing  and  evaluation 


7  From  September  2006  to  April  2007,  the  RTI  programme  was  funded  by  BAE  Systems,  with  a 
contract  from  the  MoD  only  following  in  May  2007.  This  required  BAE  Systems  and  the  MoD  to 
partner  closely,  with  trust  essential  to  enable  this. 

8  Currently  intended  to  be  the  F-35C  version,  previously  the  B,  STOVL  version. 
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and  subsequent  support  of  the  targeting  system  in  UK  service. 


In  this  environment  of  technical,  contractual,  organizational,  and  operational 
complexity,  with  a  multitude  of  interactions  between  different  factors  affecting  O&S,  it 
is  very  difficult  to  know  how  contractors  can  plan  and/or  profit  from  O&S  activities, 
without  adding  cost  upfront  (or  locking  it  in  for  later),  due  to  the  difficulties  of 
estimation  that  such  uncertainty  brings.  However,  it  appears  that  the  Harrier  GR.9 
case  study  does  highlight  that  it  can  be  done,  and  illustrates  some  of  the  tools  that 
are  needed.  It  is  notable  that  by  accurately  costing  their  work  in  advance,  BAE 
Systems  made  good  profits  on  their  Harrier  upgrade  work,  while  delivering 
substantially  reduced  O&S  costs  to  the  customer. 


2.  USMC  OSCAR  Update 

The  OSCAR  program  had  seen  the  first  major  use  of  COTS  computing  by  a 
U.S.  combat  aircraft,  and  was,  overall,  a  success.  However,  it  did  reveal  that,  while 
Moore’s  Law  may  allow  a  doubling  of  computer  power  every  eighteen  months,  the 
integration  and  testing  cycle  on  combat  aircraft  was  the  key  driver  of  program 
timescales  and  associated  costs  (Adams,  2002;  Hoppe  &  Winter,  1996). 

Unlike  the  UK’s  Harrier  GR.9  update,  in  the  OSCAR  program  for  the  AV-8B 
the  OFP  was  developed  as  part  of  a  modular  OFP  “family”  for  a  number  of  aircraft 
programs  (Logan,  2000).  This  meant  that  there  was  not  as  close  a  coupling  between 
the  introduction  of  the  new,  COTS  mission  computer  and  the  delivery  of  the  software 
for  the  OFP,  with  the  latter  using  C++  technologies  in  order  to  deliver  the  modularity 
required.  It  also  meant  that  the  costs  of  developing  the  OFP  were  spread  across  a 
range  of  programs. 


The  implementation  of  the  OSCAR  upgrade  was,  therefore,  superficially 
similar  to  the  Harrier  GR.9  update,  but  in  practice  they  were  carried  out  very 
differently.  A  key  problem  found  in  OSCAR  was  that  the  flight  testing  timescales 
were  the  key  driver  of  the  overall  update  cycle,  with  the  testing  cycle  taking  longer  to 
implement  than  the  rate  of  churn  in  the  delivery  of  new  generations  of  chips  (Adams, 
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2002;  Hoppe  &  Winter,  1996). 


In  addition,  the  timescales  during  which  combat  aircraft  operate,  with  the 
need  for  ongoing  support  for  decades,  is  a  major  issue  for  COTS  insertion — the 
chips  used  may  well  be  out  of  production,  and  possibly  unsupported  by  their  original 
commercial  supplier,  many  years  before  the  military  aircraft  they  are  installed  in  stop 
flying.  These  two  time-scale  issues  (testing  slowing  down  COTS  insertion,  with  use 
ensuring  COTS  chips  long-term  use  instead  of  rapid  replacement),  have  perhaps 
been  behind  the  apparent  lack  of  delivery  of  the  early  promises  of  COTS. 

In  the  OSCAR  update,  the  sharing  of  a  common  OFP  with  other  platforms 
and  the  use  of  COTS  technology  that  was  developed  separately  from  the  OFP, 
meant  that  costs  were  hard  to  identify,  but  what  is  clear  is  that  timescales,  of  both 
the  overall  program  and  of  individual  capability  upgrades,  took  much  longer  to 
implement  than  on  the  Harrier  GR.9.  The  costing  of  either  program  could  not  use 
data  from  the  other  (BAE  and  Boeing  interviews)  in  a  useful  way,  beyond  the  overall 
update  ballpark  figures  being  similar — $145  million  for  initial  OSCAR  RDT&E 
estimates  (Defense  Technical  Information  Center,  1999,  exc.  shared  OFP  costs)  and 
£150  for  the  Harrier  GR.9,  on  a  roughly  equivalent  basis  (Pryce,  2009). 

3.  Overview  of  Case  2 

With  the  changes  to  the  OFP  being  unpredictable,  an  important  way  to 
minimize  costs  on  the  Harrier  GR.9  upgrade,  and  in  ongoing  O&S  activities  such  as 
RTI,  was  to  minimize  the  time  it  took  to  implement  them.  While  this  is  a  simple 
enough  idea,  the  example  of  how  the  UK  GR.9  program  was  able  to  implement  them 
much  more  quickly  than  on  the  U.S.  OSCAR  program,  despite  the  use  of  a  similar 
computer  and  airframe,  shows  that  the  issue  of  design  lock-in  is  not  as  limiting  as 
may  be  expected.  The  flexibility  that  organizational  structures  can  allow,  to 
overcome  such  “hard”  technical  features,  as  well  as  accommodating  the 
unpredictable  changes  to  O&S  activities  that  operational  service  revealed,  is  a  key  to 
controlling  future  O&S  costs. 
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Section  III.  Discussion  and  Recommendations 


A.  Research  Questions  Answered 

This  report  seeks  to  show  that  the  causes  of  operations  and  support  costs  are 
many  and  varied.  In  Section  I,  the  following  three  main  questions  were  posed  for  the 
research: 

■  Is  it  the  case  that  Dynamic  CoPS,  such  as  combat  aircraft,  with  major 
issues  around  operations  in  many  changing  situations,  with  variable 
levels  of  use/damage,  over  many  years,  cannot  be  predictable  enough 
in  use  to  benefit  from  different  solutions  to  their  O&S  needs? 

■  Is  it  the  case  that  by  exploring  the  way  Dynamic  CoPS  are  used  and 
supported,  that  assumptions  that  we  are  constrained  by  design  lock-in 
of  costs  can  be  modified? 

■  How  can  we  design  new  equipment,  or  modify  old  equipment,  to 
benefit  from  such  an  approach? 

In  the  light  of  the  case  studies,  answering  these  questions  provides  some 
insight  into  the  significance  of  the  research  and  its  bearing  on  the  existing  literature. 

The  issue  of  predictability  of  O&S  costs  for  Dynamic  CoPS  can  be  seen  as 
depending  on  understanding  the  interaction  of  Dynamic  CoPS  with  their  wider 
environmental  context.  As  the  data  presented  in  the  first  case  study  shows,  once 
these  types  of  issues  are  identified,  they  can  illustrate  the  causation  of  the 
differences  in  O&S  figures  between  similar  platforms  used  and  supported  in  different 
contexts.  Once  these  issues  are  understood,  an  attempt  at  making  predictions  of 
other  platforms  can  be  attempted.  However,  what  needs  to  be  borne  in  mind  is  that 
the  numerical  data,  stored  in  databases  such  as  VAMOSC,  are  not  the  end  product 
that  is  used  to  produce  predictions  of  O&S  costs  for  future  systems,  as  currently 
happens  using  modeling  tools  such  as  OSCAM  to  meet  the  demands  of  DoDD 
5000.4  (DEPSECDEF,  2006).  Instead,  the  data  needs  to  be  interpreted  through  the 
prism  of  the  understanding  of  wider  environmental  issues  (this  is  illustrated  in  Figure 
4  below). 
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Regarding  the  basic  question  of  technological  lock-in  of  costs,  it  appears  that 
Reed  (1978)  and  others  who  advocated  this  view  are  not  correct.  Clearly,  patterns  of 
operational  use,  approaches  to  O&S,  and  relatively  minor  differences  between 
successive  versions  of  an  aircraft  can  have  a  significant  impact  on  O&S  activities, 
and  thereby,  on  associated  costs.  In  the  case  of  related,  relatively  simple  aircraft,  as 
with  the  Harrier  family,  this  still  allows  useful  data  to  be  gathered  on  the  effects  of 
complexity  factors  over  and  above  “parts  count”  type  estimates.  Their  relative 
similarity  allows  for  this.  However,  to  understand  and  use  this  data  in  a  meaningful 
way  requires  an  understanding,  and  a  way  of  using,  the  environmental  interactions 
that  underpin  the  data. 

With  more  technically  complex  and  higher  parts  count  aircraft  that  are 
unrelated,  it  appears  that  it  is  not  possible  to  use  data  from  one  to  predict  the  O&S 
costs  of  another — the  Tornado  and  F/A-18  comparison  shows  that  similarly  complex 
(in  parts  count  terms)  aircraft  can  have  very  different  O&S  numbers,  even  when  the 
data  is  available  and  substantial.  Here  an  understanding  of  the  wider  environment 
that  causes  many  of  the  arisings  and  operational  effects  that  underlie  O&S  costs 
may  be  of  considerable  use. 

Regarding  the  design  of  new  systems,  or  the  modification  of  old  ones,  the 
research  has  led  to  the  interesting  finding  that  it  is  inside  companies  that  much  of  the 
capability  to  use  existing  data  and  experience  lies.  It  seems  that  Dynamic  CoPS  can 
benefit  contractors  through  O&S  contracting  arrangements,  despite  their  much 
higher  levels  of  unpredictability  compared  to  static  CoPS,  with  the  cases  drawn  from 
Harrier,  at  least,  showing  that  this  is  possible.  As  such,  Davies  and  Hobday’s  (2005) 
work  may  be  applicable. 


However,  it  may  not  be  directly  applied  in  an  easily  repeatable  form  without 
understanding  the  wider  environmental  factors  embedded  in  the  experience  of 
contractors’  engineering  and  management  teams.  While  using  the  solutions 
approach  that  Davies  and  Hobday  (2005)  propose  in  O&S  support  of  combat  aircraft 
is  possible,  it  currently  depends  on  a  detailed,  in-depth,  highly  contextual  knowledge 
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of  the  wide  range  of  environmental  factors  that  lead  to  O&S  costs,  and  the  variance 
caused  by  relatively  minor  changes  in  the  interactions  between  the  system  and  its 
wider  environments  on  these  costs. 

1.  Case  Studies  Review 

The  findings  throw  into  question  the  idea  of  using  past  data  to  project  future 
costs  of  new  systems  based  on  simple  assumptions  of  similarity  of  role,  parts  count, 
or  equivalent  weight,  and  so  forth  (i.e.,  approaches  that  are  based  on  the  innate 
characteristics  of  a  platform  or  system).  If  there  are  significant  differences  in  the 
O&S  costs  and  the  causes  of  the  costs,  between  similar  platforms,  then  it  is 
essential  that  the  causes  of  these  differences  are  understood  in  detail  before  being 
applied  to  future  designs.  A  number  of  factors  that  are  external  to  the  system  that 
affect  its  O&S  costs  need  to  be  taken  into  account,  too.  It  may  be  that  the  future 
design  is  particularly  susceptible  to  some  particular  issue  that  is  “lost  in  the  noise”  of 
aggregated  data  derived  on  a  systems  based  view,  but  which  may  be  clearly 
illustrated  if  we  look  at  wider  environmental  factors  that  lie  outside  the  platform. 

A  case  in  point,  given  in  the  case  studies,  is  the  operation  of  UK  Sea  Harrier 
aircraft  from  ski-jump  equipped  aircraft  carriers.  The  fact  that  one  of  these  ships 
caused  damage  to  aircraft  undercarriage  units  was  not  catastrophic  in  this  case  but, 
in  large  part,  this  was  due  to  the  undercarriage  being  of  robust  design,  thanks  to 
very  different  original  requirements.  If  the  undercarriage  had  been  designed  by  the 
assumed  loads  for  the  ski-jump,  modeled  as  part  of  the  design  and  clearance 
program,  it  could  well  have  failed  in  service  use,  leading  to  expensive  re-design,  re¬ 
manufacture,  and  modification  work  for  the  entire  fleet,  or  to  the  aircraft  carriers.  If 
the  simple,  baseline  assumptions  of  the  nature  of  ski-jump  ramp  design  had  been 
widened  to  look  at  possible  worst  case  scenarios,  the  issue  may  have  been 
accounted  for  earlier,  and  its  costs  would  not  have  come  as  a  surprise. 

Similarly,  the  Harrier  GR.9  case  illustrates  how,  despite  minor  overt 
differences  from  the  AV-8B  as  a  platform,  the  mission  system  upgrade  was  carried 
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out  via  quicker  testing  cycles,  leading  to  lower  costs  than  might  otherwise  have  been 
incurred.  This  was  enabled  by  the  use  of  very  different  managerial  and 
organizational  structures,  as  well  as  the  different  use  of  software  tools.  Such  specific 
differences  between  two  apparently  similar  cases  would  need  to  be  understood 
before  planning  and  costing  the  system  upgrade  of  another  platform  similar  to  the 
Harrier,  with  the  O&S  infrastructure  to  deliver  the  update,  and  the  capture  of  any 
data  to  be  used  to  plan  the  work  and  to  estimate  costs,  needing  to  be  understood  on 
the  basis  of  the  range  of  variance  achieved  between  Harrier  variants. 

This  use  of  a  concept  of  a  wider  environment  as  the  context  from  which  O&S 
costs  derive,  via  interactions  of  the  systems  and  platforms  in  use  by  the  armed 
services  with  this  wider  environment,  lies  at  the  heart  of  the  recommendations  to  be 
made  in  this  report.  However,  the  example  of  the  Harrier  updates  also  shows  how 
this  understanding  of  the  environmental  context  can  also  benefit  RDT&E.  This 
means  that  the  concept  of  wider,  environmental  interaction  could  also  have  a  place 
in  the  re-definition  of  complexity  being  carried  out  by  DARPA  under  the  aegis  of 
META. 


Building  on  these  findings  lies  at  the  heart  of  the  ongoing  research  program 
that  this  report  is  based  on.  With  a  clear  idea  of  the  effect  of  all  the  factors,  and  their 
interactions,  that  cause  O&S  issues,  and  their  related  costs,  it  is  thought  that  a  more 
useful  method  of  applying  data  from  existing  programs  to  future  ones  can  be 
developed. 

2.  Recommendation  for  a  New  Costing  Method 

This  report  has  shown  that  in  order  to  understand  all  the  factors  that  lead  to 
O&S  costs,  we  need  to  look  beyond  the  scope  of  systems  and  platform  and  their 
innate  characteristics.  We  need  to  understand  all  the  interactions  between  a 
Dynamic  CoPS-type  system  and  its  operating  and  support  environment  in  order  to 
fully  understand  what  its  O&S  may  cost. 
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In  order  to  apply  this  understanding  to  the  costing  of  new  systems,  it  is 
considered  that  a  new  approach  to  costing  O&S,  beyond  the  analytically  based 
approach  spelled  out  in  DoDD  5000.4  (DEPSECDEF,  2006)  should  be  pursued.  An 
overview  of  the  current  and  proposed  approaches  is  given  in  Figure  4. 


Figure  4.  Current  and  Proposed  Approaches  to  O&S  Costing 

In  order  to  implement  this  new  approach,  it  is  recommended  that  for  each 
new  system,  the  interactions  of  not  just  components  in  the  system,  but  of  these 
components  and  the  system  as  a  whole  with  its  wider  environment,  be  examined. 

While  DARPA,  under  the  META  initiative,  is  looking  at  interactions  between 
system  components  for  RDT&E,  this  proposed  new  approach  to  costing  in  O&S 
would  look  at  a  broader  range  of  factors  than  META,  but  would  do  so  in  a  less 
defined  way.  This  is  because  for  systems  under  design,  the  level  of  O&S  support  is 
very  hard  to  pin  down  in  hard  numerical  terms,  as  the  patterns  of  operational  use, 
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the  nature  of  the  support  system,  the  approaches  used  by  contractors,  and  so  forth, 
would  be  very  hard  to  define  when  the  system  is  at  an  early  stage. 

However,  as  the  research  for  this  report  has  shown,  the  wider  environment  of 
patterns  of  use,  support  structures,  and  so  forth,  do  allow  significant  scope  for  the 
control  of  O&S  costs.  If  the  nature  and  significance  of  the  interactions  with  the 
environment  can  be  identified  for  existing  systems,  and  compared  to  the  likely 
occurrence  or  significance  of  similar  interactions  for  new  systems,  then  it  is  possible 
that  the  range  of  variance  in  likely  O&S  costs  from  those  generated  by  the  methods 
defined  under  DoDD  5000.4  (DEPSECDEF,  2006)  can  be  estimated. 

The  basic  concept  would  be  to  generate,  for  each  environmental  interaction,  a 
range  of  costs  for  a  given  set  of  capabilities,  depending  on  whether  or  not  the 
environmental  interactions  are  expected  to  generate  costs  higher  or  lower  than 
those  derived  from  the  DoDD  5000.4  originated  baseline  cost  figure.  A  simplified 
overview  is  given  in  Figure  5. 


Capability 


Figure  5.  The  Effects  of  Costing  Based  on  Environment 

Note.  Env.  =  Environmental  interaction  impact  on  costs. 
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A  possible  example  of  how  this  might  work,  derived  from  the  case  studies 
described  previously,  would  be  the  cost  of  the  capability  of  the  targeting  system  of 
the  Joint  Strike  Fighter.  Using  data  from  previous  targeting  systems,  and  comparing 
on  the  basis  of  component  counts  and  lines  of  code,  essentially  as  in  DoDD  5000.4 
(DEPSECDEF,  2006)  based  approaches  and  in  the  initial  approach  given  by 
DARPA’s  META,  would  give  the  baseline  figure,  shown  in  blue  in  Figure  5.  This 
would  vary  according  to  the  capability  desired  from  the  targeting  system. 


Adding  in  environmental  interaction  factors  from  the  case  studies,  we  could 
see  that  the  experience  of  the  Rapid  Technology  Insertion  approach  used  on  the 
Harrier,  with  its  environment  of  reducing  cycle  times  and  costs  for  the  testing  and 
integration  of  a  new  system  on  the  aircraft,  and  its  functioning  in  a  service 
environment  (e.g.,  on  board  ship  or  in  dispersed,  STOVL  operations),  could  benefit 
the  overall  cost  estimate.  This  would  give  a  lower  figure  than  the  baseline  one, 
shown  by  the  green  line  in  Figure  5. 

However,  the  consideration  of  other  environmental  factors  could  adversely 
affect  the  O&S  costs  of  the  targeting  system.  Again,  based  on  the  case  studies,  the 
effects  of  the  different  loads  experienced  by  the  nose  undercarriage  on  the  STOVL 
and  carrier  (B  and  C)  versions  of  the  F-35  JSF  may  impact  the  targeting  system,  as 
it  is  installed  near  the  nose  undercarriage  unit,  and  vibrations,  impacts,  and  so  forth, 
may  adversely  affect  it.  Although  the  baseline  costs  will  assume  that  the  vibrations 
and  other  factors  are  as  stated  in  specifications  for  vibration  given  by  national  design 
standards  (Beier,  1987),  the  unpredictability  of  vibration  in  STOVL  operations,  or  the 
vibrations  and  shocks  caused  by  dispersed  operations  or  uneven  deck  surfaces, 
may  exceed  these  vibration  levels  and  cause  higher  than  expected  failure  in  the 
targeting  system.  Consideration  of  these  factors  may  produce  the  red  line  of  O&S 
costs  in  Figure  5,  where  costs  are  higher  than  the  baseline. 


In  order  to  carry  out  these  assessments  of  environmental  factors,  modeling 
the  various  interactions  experienced  or  expected  would  need  to  be  carried  out  for 
each  type  of  interaction,  and  the  relationship  between  each  modeled  interaction  type 
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would  also  need  to  be  related.  The  models  would  need  to  consider  the  nested 
interactions  between  a  component,  the  wider  system,  and  the  operational  and 
support  environments.  Figure  6  shows  a  generic  process  model  and  two  versions  of 
the  model  from  the  JSF  targeting  pod  example  given. 

The  cost  modeling  would  be  carried  out  using  actual  data  from  current 
systems,  with  the  environmental  data  cross  read  between  old  and  new  systems,  with 
components  and  systems  data  generated  using  existing  (DoDD  5000.4; 
DEPSECDEF,  2006)  or  newly  developed  (e.g.,  META)  approaches.  It  would  be 
necessary  to  control  for  different  levels  of  technology,  such  as  when  using  data  from 
metal  structures  to  model  those  made  of  composite  material. 


Carrier  deck 


Targeting  system 


Environment 


System 


Component 


Undercarriage 


1 

RTI  management 

Targeting  system 

Undercarriage 

Figure  6.  Interaction  Models  for  Costing  O&S 
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3.  Implementation  of  Findings  and  Further  Research 

The  research  reported  here  has  shown  that  there  is  scope  for  changing  the 
costs  of  O&S  of  complex  systems  such  as  Dynamic  CoPS.  While  the  RDT&E  costs 
may  be  locked  in  by  the  overall  design  of  the  system,  there  is  much  greater  scope 
for  controlling  costs  in  O&S. 

The  research  has  taken  place  with  companies  who  have  considerable 
experience  in  O&S  activities,  supporting  systems  they  have  designed,  as  well  as 
ones  designed  by  others.  While  this  has  revealed  the  extent  of  the  knowledge  and 
experience  of  the  contractors  concerned  in  understanding  O&S,  in  advance  of 
government  costing  processes  such  as  DoDD  5000.4  (DEPSECDEF,  2006),  the 
research  has  also  proposed  an  outline  methodology  that  would  allow  government 
costing  activities  to  benefit  from  the  understanding  gained  from  contractors’ 
experiences. 

It  is  proposed  that  in  order  to  test  out  this  costing  methodology,  that  further 
work  be  carried  out  with  contractors  in  order  to  see  if  it  is  practical.  A  proposal  that 
features  this  activity  as  part  of  its  work  plan  has  been  accepted  for  funding  by  NPS 
under  call  number  NPS-BAA-1 1-002,  under  the  title  Business  Models  for  Cost 
Sharing  &  Capability  Sustainment,  and  will  be  reported  in  mid-2012.  In  particular,  the 
importance  of  a  flexible,  nuanced,  and  in-depth  understanding  of  the  cost  estimates 
for  O&S,  without  which  business  models  to  carry  it  out  cannot  be  sustained,  will  be 
explored  in  that  work. 

The  empirical  work  reported  here  has  led  to  the  belief  that  the  costing 
approach  proposed  is  indeed  feasible,  as  it  is  a  more  formalized  statement  of  how 
contractors  currently  operate.  If  it  can  be  shown  that  this  approach  works  in  a  wider 
setting,  for  example,  in  government  costing  communities,  then  this  research  will 
have  advanced  the  state  of  the  art  in  O&S  costing  methods.  No  doubt  this  will 
involve  considerable  further  effort,  but  in  the  meantime,  the  author  is  happy  to  hear 
from,  discuss  with,  and  present  to  colleagues  and  researchers  in  similar  fields  in 
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order  to  further  understand  this  constantly  changing,  and  constantly  fascinating, 
subject. 
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